Lecture 7 : Channel coding theorem for discrete - time continuous memoryless channel

Let us first define, for the random sequences X = [X1, . . . , Xn] and Y = [Y1 . . . , Yn] and their corresponding sequence realizations x = [x1, . . . , xn] and y = [y1 . . . , yn] where xk, yk ∈ R, the following probability density functions (pdfs): fX(x) ∆ = ∏n k=1 fX(xk) as the input pdf, fY |X(y|x) = ∏n k=1 fY |X(yk|xk) as the memoryless channel (transition) pdf, and fY (y) as the output pdf. Each element Xk (or just X without the index k) are independent and identically distributed i.i.d. Further, due to the i.i.d. property of X and the specified memoryless property imposed on the channel transition pdf, we have fY (y) = ∏n k=1 fY (yk). Define the so called and the mutual information quantity Let I(X;Y ) , ∫